Robust and stealthy video watermarking into regions of successive frames

ABSTRACT

An implementation of a technology is described herein for the protection of rights in the content of a video sequence. This technology further generally relates to a technology facilitating embedding imperceptible, de-synchronization-resistant watermarks in video sequence and facilitating detecting such watermarks. These watermarks are resistant against unintentional and intentional modifications. In particular, the watermarks are resistant to de-synchronization. In addition, the watermarks are perceptually invisible. The watermarks are hidden in the video so that flicker is minimized or eliminated. More specifically, this technology hides a watermark (of portions thereof) over one or more regions of successive frames. Each region has a center defined by a hash value. A watermark (of portions thereof) is encoded into the region in a “plateau” manner. The mark (of portions thereof) is fully encoded in the frames surrounding the region&#39;s center, but trail off towards the edges of the region. This abstract itself is not intended to limit the scope of this patent. The scope of the present invention is pointed out in the appending claims.

TECHNICAL FIELD

This invention generally relates to a technology facilitating theprotection of rights in the content of a video sequence. This inventionfurther generally relates to a technology facilitating embeddingimperceptible, de-synchronization-resistant watermarks in video sequenceand facilitating detecting such watermarks.

BACKGROUND

A “video sequence” is a series of images that typically capture (orsimulate) motion, life, action, movement, etc. The video sequences aretypically accompanied by audio. Watermarking a video sequence presents aseries of significant challenges that are greater than those faced whenwatermarking other “digital goods.”

“Digital goods” is a generic label for electronically stored ortransmitted content. Examples of digital videos include images, audioclips, video, digital film, multimedia, software, and data.

A video sequence is a specific type of digital videos. It may also becalled a “digital video,” “video signal,” “video bitstream,” “videostream,” “streaming video,” “video media,” “video object,” “video,”“digital film,” “digital movie,” and the like. The emerging field of“digital film” is a high-quality form of video.

Digital videos are often distributed to consumers over private andpublic networks—such as Intranets and the Internet. In particular, theymay be “broadcast” via streaming video of a live or recorded event. Inaddition, these videos are distributed to consumers via fixed computerreadable media, such as a compact disc (CD-ROM), digital versatile disc(DVD), soft magnetic tape, soft magnetic diskette, or hard magnetic disk(e.g., a preloaded hard drive).

Digital videos may be stored in one or many different formats. Some ofthe more common multimedia file formats include: MPEG, Video ofWindows®, QuickTime™, RealVideo™, Shockwave™, and the like.

Unfortunately, it is relatively easy for a person to pirate the pristinedigital content of a digital video at the expense and harm of thecontent owners. Content owners include the content author, artist,publisher, developer, distributor, etc. The content-based industries(e.g., entertainment, music, film, television, etc.) that produce anddistribute content are plagued by lost revenues due to digital piracy.

Modern digital pirates effectively rob content owners of their lawfulcompensation. Unless technology provides a mechanism to protect therights of content owners, the creative community and culture will beimpoverished.

Watermarking

Watermarking is one of the most promising techniques for protecting thecontent owner's rights of a digital video. Generally, watermarking is aprocess of altering the digital video such that its perceptualcharacteristics are preserved. More specifically, a “digital watermark”(or simply “watermark”) is a pattern of bits inserted into a digitalvideo that may be used to identify the content owners and/or theprotected rights.

Watermarks are designed to be completely invisible or, more precisely,to be imperceptible to humans and statistical analysis tools. Ideally, awatermarked video signal is perceptually identical to the original videosignal.

A watermark embedder (i.e., encoder) embeds a watermark into a digitalvideo. It typically uses a secret key to embed the watermark. Awatermark detector (i.e., decoder) extracts the watermark from thewatermarked digital video.

To detect the watermark, some watermarking techniques require access tothe original unmarked digital video or to a pristine specimen of themarked digital video. Some, watermarking techniques are “blind.” Thismeans that they do not require access to the original unmarked digitalvideo or to a pristine specimen of the marked digital video. Of course,these “blind” watermarking techniques are desirable when the watermarkdetector is publicly available.

Before detection, a watermarked signal may undergo many possible changesby users and by the distribution environment. These changes may includeunintentional modifications, such as noise and distortions. Moreover,the marked signal is often the subject of malicious attacks particularlyaimed at disabling the detection of the watermark.

Ideally, a watermarking technique should embed detectible watermarksthat resist modifications and attacks as long as they result in signalsthat are of perceptually the same quality. A watermarking technique thatis resistant to modifications and attacks may be called “robust.”Aspects of such techniques are called “robust” if they encourage suchresistance.

Generally speaking, a watermarking system should be robust enough tohandle unintentional noise introduction into the signal (such noise mybe introduced by A/D and D/A conversions, compressions/decompressions,data corruption during transmission, etc.)

Furthermore, a watermarking system should be robust enough and stealthyenough to avoid purposeful and malicious detection, alternation, and/ordeletion of the watermark. Such attack may use a “shotgun” approachwhere no specific watermark is known or detected (but is assumed toexist) or may use “sharp-shooter” approach where the specific watermarkis attacked.

Those of ordinary skill in the art are familiar with conventionaltechniques and technology associated with watermarks, watermarkembedding, and watermark detecting. In addition, those of ordinary skillin the art are familiar with the typical problems associated with properwatermark detection after a marked signal has undergone changes (e.g.,unintentional noise and malicious attacks).

Herein, such a digital watermark may be simply called a “watermark.”Generically, it may be called an “information pattern of discretevalues.”

Desiderata of Watermarking Technology

Watermarking technology has several highly desirable goals (i.e.,desiderata) to facilitate protection of copyrights of video contentpublishers. Below are listed several of such goals.

Perceptual Invisibility. The embedded information should not induceperceptual changes in the video quality of the resulting watermarkedsignal. The test of perceptual invisibility is often called the “goldeneyes and ears” test.

Statistical Invisibility. The embedded information should bequantitatively imperceptive for any exhaustive, heuristic, orprobabilistic attempt to detect or remove the watermark. The complexityof successfully launching such attacks should be well beyond thecomputation power of publicly available computer systems. Herein,statistical invisibility is expressly included within perceptualinvisibility.

Tamperproofness. An attempt to remove the watermark should damage thevalue of the video well above the hearing threshold.

Cost. The system should be inexpensive to license and implement on bothprogrammable and application-specific platforms.

Non-disclosure of the Original. The watermarking and detection protocolsshould be such that the process of proving video content copyright bothin-situ and in-court, does not involve usage of the original recording.

Enforceability and Flexibility. The watermarking technique shouldprovide strong and undeniable copyright proof. Similarly, it shouldenable a spectrum of protection levels, which correspond to variablevideo presentation and compression standards.

Resilience to Common Attacks. Public availability of powerful digitalvideo editing tools imposes that the watermarking and detection processis resilient to attacks spawned from such consoles.

Hard-to-Break. A watermark is “hard-to-break” when it is “extremelyhard” for an attacker to break the watermark even though the attackermay know watermarking technique. Here, “breaking” refers to successfullymodifying or removing the watermark. In particular, it should be nearlyimpossible to break the mark under almost all practical situations evenif an attacker has a supercomputer.

Watermark Circumvention

In general, there are two common classes of malevolent attacks:

-   -   1. De-synchronization of watermark in digital video signals.        These attacks alter video signals in such a way to make it        difficult for the detector to identify the location of the        encoded watermark codes.    -   2. Removing or altering the watermark. The attacker discovers        the location of the watermark and intentionally alters the video        clip to remove or deteriorate a part of the watermark or its        entirety.        Particular Video Watermarking Challenges

A video is a series of video “frames.” Each frame of the video is animage. Since videos are a series of images, one way to watermark a videois to embed a watermark (wholly or partially) in each frame (or asignificant number) of the video.

As mentioned earlier, watermarking a video sequence presents a series ofsignificant challenges that are greater than those faced whenwatermarking other “digital goods.” Particular examples of thesechallenges include perceptual invisibility and resistance tode-synchronization attacks. Although watermarking other types of media(e.g., images and audio) also faces these challenges, the problems ofperceptual invisibility and resistance to de-synchronization areparticularly acute and specifically unique for videos.

De-Synchronization Attacks

The watermark (or portions thereof) may be embedded into each frame ofthe video. However, the chances of a digital pirate discovering thewatermark increases as the watermark repetition increases. Embedding thewatermark (or portions thereof) in each frame is also undesirablebecause it provides convenient range for the pirate to focus herefforts. In addition, it provides potentially thousands of boundedtargets (i.e., frames) containing the same hidden data (i.e., thewatermark). With this much bounded information, a digital pirate has agood chance of determining the watermark.

To overcome this problem, watermarks (or portions thereof) may beselectively encoded in individual frames or groups of frames within thevideo. To find the encoded information later, the detector typicallymust be synchronized along the temporal axis so that it know where (orwhen) to look for the watermarks. Digital pirates know this. Ade-synchronization attack is one of their most watermark-fatal arrows intheir quiver. In addition, de-synching may occur unintentionallyparticularly when video signal is transmitted.

Resisting de-synchronization is a particularly difficult challenge inthe video realm. A pirate may, for example, do any of the following tode-synch a video:

-   -   remove frames;    -   add new frames (such as commercials);    -   add copied frames (copies of adjacent frames);    -   change frames/sec rate;    -   rearrange frames.

If this de-synch attack splits a series of frames in which the fuillwatermark is encoded, then the watermark may go undetected. If thisattack manages to remove the isolated frames including the watermarks,then the watermark may go undetected.

Perceptual Invisibility

As mentioned above, a watermark should be perceptually invisible (whichinclude statistically invisible) within the signal. Achieving perceptualinvisibility is a particularly difficult challenge in the video realm.

Typically, a series of successive frames have one or more commonsections. These common sections contain the same image data. Forexample, if the camera capturing the video frames is fixed on relativelystationary objects or people, then the vast majority of each frame willbe identical. Typically, if the camera is fixed, the background remainsidentical in each frame.

If the watermark (or portions thereof) is not encoded in every frame ofthe video, then some frames will have no portion of the watermarkencoded therein. Consequently, there will be a transition betweenencoded frames and non-encoded frames. Typically, perceptible “flicker”occurs at that transition. Flicker is the perceptible manifestation ofthe transition. This problem is particular to video.

Flicker may be visible to the human eye. If not, it may be noticeable bystatistical analysis tools. Since watermark encoding introduces “noise”into a frame, the transition from “noisy” to “noiseless” frame producesperceptible flicker in the common sections of the frames of thattransition.

Armed with the knowledge of flickering, a digital pirate can focus herattack on the frames in and around transitions.

Framework to Thwart Attacks

Accordingly, there is a need for a new framework for hiding anddetecting watermarks in digital video signals that is effective againstunintentional and intentional modifications. In particular, theframework should be resistant to de-synchronization. The frameworkshould possess several attributes that further the desiderata ofwatermark technology, described above. In particular, it should beperceptually invisible; thus, it should minimize or eliminate flicker.

SUMMARY

Described herein is a technology facilitating the protection of rightsin the content of a video sequence. This technology further generallyrelates to a technology facilitating embedding imperceptible,de-synchronization-resistant watermarks in video sequence andfacilitating detecting such watermarks.

One or more implementations, described herein, hide and/or detectstealthy and robust watermarks in digital video signals. Thesewatermarks are resistant against unintentional and intentionalmodifications. In particular, the watermarks are resistant tode-synchronization. In addition, the watermarks are perceptuallyinvisible. The watermarks are hidden in the video so that flicker isminimized or eliminated.

More specifically, one or more implementations, described herein, hide awatermark (of portions thereof) over one or more regions of successiveframes. Each region has a center defined by a hash value. A watermark(of portions thereof) is encoded into the region in a “plateau” manner.The mark (of portions thereof) is fully encoded in the framessurrounding the region's center, but trail off towards the edges of theregion.

This summary itself is not intended to limit the scope of this patent.Moreover, the title of this patent is not intended to limit the scope ofthis patent. For a better understanding of the present invention, pleasesee the following detailed description and appending claims, taken inconjunction with the accompanying drawings. The scope of the presentinvention is pointed out in the appending claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The same numbers are used throughout the drawings to reference likeelements and features.

FIG. 1A illustrates the frames (i.e., images) of a video sequence.

FIG. 1B is a diagram illustrating a “plateau” shape of the watermarkingof a sequence region; this illustrates an example of a watermarkedsequence region in accordance with an implementation of the inventionherein.

FIG. 1C is a diagram illustrating multiple plateau-shaped sequenceregions of a video signal; this illustrates an example of multiplewatermarked sequence regions of a signal which is in accordance with animplementation of the invention herein.

FIG. 2 is a schematic block diagram showing a video watermarkingarchitecture in accordance with an implementation of the inventionherein.

FIG. 3 is a schematic block diagram showing a video watermark embeddingsystem in accordance with an implementation of the invention herein.

FIG. 4 is a flow diagram showing an illustrative methodologicalimplementation (e.g., video watermark embedding) of the inventionherein.

FIG. 5 is a schematic block diagram showing a video watermark detectingsystem in accordance with an implementation of the invention herein.

FIG. 6 is a flow diagram showing an illustrative methodologicalimplementation (e.g., video watermark detecting) of the inventionherein.

FIG. 7 is an example of a computing operating environment capable ofimplementing an implementation (wholly or partially) of the inventionherein.

DETAILED DESCRIPTION

In the following description, for purposes of explanation, specificnumbers, materials and configurations are set forth in order to providea thorough understanding of the present invention. However, it will beapparent to one skilled in the art that the present invention may bepracticed without the specific exemplary details. In other instances,well-known features are omitted or simplified to clarify the descriptionof the exemplary implementations of present invention, thereby betterexplain the present invention. Furthermore, for ease of understanding,certain method steps are delineated as separate steps; however, theseseparately delineated steps should not be construed as necessarily orderdependent in their performance.

The following description sets forth one or more exemplaryimplementations of Robust and Stealthy Video Watermarking thatincorporate elements recited in the appended claims. Theseimplementations are described with specificity in order to meetstatutory written description, enablement, and best-mode requirements.However, the description itself is not intended to limit the scope ofthis patent.

The inventors intend these exemplary implementations to be examples. Theinventors do not intend these exemplary implementations to limit thescope of the present invention. Rather, the inventors have contemplatedthat the present invention might also be embodied and implemented inother ways, in conjunction with other present or future technologies.

An example of an embodiment of Robust and Stealthy Video Watermarkingmay be referred to as an “exemplary video watermaker.”

Incorporation by Reference

The following co-pending patent applications are incorporated byreference herein (which are all assigned to the Microsoft Corporation):

-   -   U.S. patent application Ser. No. 09/390,271, entitled “A        Technique for Watermarking an Image and a Resulting Watermarked'        Image” filed Sep. 7, 1999;    -   U.S. patent application Ser. No. 09/390,272, entitled “A        Technique for Detecting a Watermark in a Marked Image” filed on        Sep. 7, 1999;    -   U.S. patent application Ser. No. 09/316,899, entitled “Audio        Watermarking with Dual Watermarks” filed on May 22, 1999;    -   U.S. patent application Ser. No. 09/614,660, entitled “Improved        Stealthy Audio Watermarking” filed on Jul. 12, 2000;    -   U.S. patent application Ser. No. 09/843,234, entitled “Robust        Recognizer of Perceptually Similar Content” filed on Apr. 24,        2001;    -   U.S. patent application Ser. No. 09/843,279, entitled        “Derivation and Quantization of Robust Non-Local Characteristics        for Blind Watermarking” filed on Apr. 24, 2001;    -   U.S. patent application Ser. No. 09/843,254, entitled        “Recognizer of Audio-Content in Digital Signals” filed on Apr.        24, 2001; and    -   U.S. patent application Ser. No. 09/421,986, entitled “System        and Method for Hashing Digital Images” filed on Oct. 19, 1999.        Introduction

The one or more exemplary implementations, described herein, of thepresent invention may be implemented (whole or in part) by a videowatermarking architecture 200 and/or by a computing environment likethat shown in FIG. 7.

Herein, references to embedding or detecting a watermark expresslyincludes the embedding or detecting of a portion of a watermark.Portions of the watermark may, for example, be embedded in a singleframe. The collection of such frames may constitute the entirewatermark. Moreover, references herein to watermarks expressly includeany information data patterns.

With the exemplary video watermaker, the watermarks are encoded (e.g.,embedded) over regions of successive frames. These regions includesuccessive adjacent frames. These regions may also be called “temporalregions” since the frames typically fall into a specific time sequencein the video signal. Herein, they may also be called “sequence regions”and “neighbor regions.”

A marked video includes multiple regions. The centers of each areapparently randomly selected. The watermark is encoded into the regionin a “plateau” manner. The region is approximately plateau shaped. Thismeans that the watermark is fully encoded in the frames surrounding theregion's center, but trail off towards the edges of the region.

Typically, the watermark detection of the exemplary video watermakerapproximately locates the center of each region. Since the watermark isencoded over a region of frames (rather than isolated single frames),the watermark detection system can find the embedded mark anywherewithin the region. This is true even when it fails to locate the exactcenter of the region because of de-synchronization. In context ofsequence regions that are approximately plateau shaped, they may becalled “plateau regions,” “watermark plateaus,” and the like.

Described herein, the exemplary video watermaker has at least twoapproaches: a “fully blind” and “partially blind” approach.

Hashing

Generally, hashing techniques are used in many areas such as databasemanagement, querying, cryptography, and many other fields involvinglarge amounts of raw data.

In general, a hashing technique maps a large block of raw data intorelatively small and structured set of identifiers. These identifiersare also referred to as “hash values” or simply “hash.” By introducing aspecific structure and order into raw data, the hashing functiondrastically reduces the size of the raw data into short identifiers. Itsimplifies many data management issues and reduces the computationalresources needed for accessing large databases.

Mathematically, a hashing technique includes a hashing function H(·).That function takes a signal x as input and computes a short vectorh=H(x). That vector is an apparently random value in some large set.That vector h is a hash value.

Alternatively, the hashing technique may employ a secret key K. Thiscryptographic hashing technique includes a hashing function H_(K)(·).That function takes a signal x as input and computes a short vectorh=H_(K)(x). That vector (i.e., hash value) is an apparently random valuein some large set, but it is indexed by a secret key K.

A hash value may be thought of as a binary string given a signal (e.g.,an image). This string serves as the signature of the input signal andis approximately invariant under all acceptable modifications on theinput signal (i.e., modifications under which the quality of the inputimage is preserved for all practical purposes). To be more precise, thehash technique used by the exemplary video watermaker operates on frames(i.e., images) of a video signal such that:

-   -   The hash values possess approximate uniform distribution;    -   The hash values of two perceptually distinct signals (e.g.,        images) are approximately independent; and    -   The hash values of two perceptually similar signals (e.g.,        images) are the same with high probability.

Herein, perceptual similarity may be thought of in this manner: If twosignals (e.g., images) are perceptually similar, then an observer shouldbe able to say that they are indeed the same signals (e.g., images) forall practical purposes, such that a reasonable amount of perceptualquality is maintained. Such perceptual qualities also applies tostatistical analysis of machines as well as humans.

In addition, for the purpose of identifying the location of thewatermark in a video signal, an intermediate hash function servesequally well as a final hash function. See the descriptions in theincorporated pending U.S. patent applications (in particular, the onesentitled “Robust Recognizer of Perceptually Similar Content” and“Recognizer of Audio-Content in Digital Signals”) for more details onintermediate and final hashing.

When selecting an intermediate hash function, the hash values ofperceptually distinct signals are distant from each other (in the senseof d(. , .)) and hash values of perceptually similar signals are closeto each other (in the sense of d(. , .)). Herein, d(. , .) refers tonormalized Hamming distance.

For more information about hashing techniques that may be employed withthe exemplary video watermaker, see the following pending U.S. patentapplications (which are incorporated by reference):

-   -   Ser. No. 09/843,234, entitled “Robust Recognizer of Perceptually        Similar Content” filed on Apr. 24, 2001;    -   Ser. No. 09/843,254, entitled “Recognizer of Audio-Content in        Digital Signals” filed on Apr. 24, 2001; and    -   Ser. No. 09/421,986, entitled “System and Method for Hashing        Digital Images” filed on Oct. 19, 1999.        Exemplary Video Watermarking Overcomes the Particular Challenges

As mentioned earlier, watermarking a video sequence presents a series ofsignificant challenges that are greater than those faced whenwatermarking other digital goods. Particular examples of thesechallenges include perceptual invisibility and resistance tode-synchronization. Although watermarking other types of media (e.g.,images and audio) also faces these challenges, the problems ofperceptual invisibility and resistance to de-synchronization areparticularly acute and specifically unique for videos.

De-Synchronization

The watermark (or portions thereof) may be embedded into each frame ofthe video. However, the chances of a digital pirate discovering thewatermark increases as the watermark repetition increases. Embedding thewatermark (or portions thereof) in each frame is also undesirablebecause it provides convenient range for the pirate to focus herefforts. In addition, it provides potentially thousands of boundedtargets (i.e., frames) containing the same hidden data (i.e., thewatermark) that the pirate may attack. With these narrow targets, adigital pirate has a good chance of determining the watermark.

To overcome this problem, the exemplary video watermaker selectivelyencodes watermarks (or portions thereof) into sequence regions withinthe video. To find the encoded information later, the exemplary videowatermaker approximately locates the centers of these regions by hashingthe video signal.

If a de-synch attack removes, adds, or rearranges frames, the centerdetermined by the detection system of the exemplary video watermakeroffset from the true center of the region. Since the multiple frameswithin that region are encoded with the watermark, the exemplary videowatermaker can detect the watermark in a frame offset from the center.The de-synch would need to significantly alter the video signal beforethe center determined by the detection system of the exemplary videowatermaker is offset enough to be miss the region entirely.

Perceptual Invisibility

The exemplary video watermaker encodes a watermark that is perceptuallyinvisible within the signal. It minimizes or eliminates flicker causedby watermark encoding.

The watermark is encoded into the sequence region in a “plateau” manner.This means that the watermark is fully encoded in the frames surroundingthe region's center, but trail off towards the edges of the region.

The trailing edges of the plateau give the region boundaries a softertransition between marked frames and unmarked frames. This issignificantly less perceptible—statistically and visually—than the“flicker” of traditional video watermarking.

Plateau-Shaped Sequence Regions

Video Sequence Example

FIG. 1A illustrates an example of a series of individual images (i.e.,frames) that collectively compose a video sequence over time. Thisexample video sequence includes frames 140. These frames are labeledx_(i), where i=1, 2, . . . , n. The numbered order of the frames (1, 2,. . . , n) indicates the typical chronological order of the frames. Thisvideo sequence may be all of or any portion of a complete video package.

Plateau-Shaped Watermarked Region

FIG. 1B illustrates a diagram of a plateau-shape of the watermarkedsequence region. This illustrates an example of a watermarked sequenceregion in accordance with the exemplary video watermaker. The“plateau-shaped” watermark encoding technique of the exemplary videowatermaker reduces (or eliminates) the effect of de-synchronization andflicker.

The frames 140 are indicated by the same x_(i) labeling as FIG. 1A.Dashed horizontal line 142 indicates a baseline of a video signalwithout watermark encoding. Solid line 144 indicates the actual videosignal relative to the baseline 142. A video signal—such as the oneindicated by line 144—is typically not perfectly smooth, straight, etc.For the sake of simplicity, video signal 144 is illustrated in FIG. 1Bin an idealized form.

In accordance with the exemplary video watermaker, an informationpattern (such as a watermark or a portion thereof) is embedded into thevideo signal 144. This may be accomplished using traditional or newwatermarking techniques. Image watermarking techniques may be used toembed watermarks into the frames of the video signal.

Within a range of frames, the watermark is embedded. That range offrames is called the plateau region 150. The plateau region ranges fromframe x_(j−k) to x_(j+k) in FIG. 1B, where j is the center of the regionand 2k+1 is the length of the region. The center frame of the region 150is frame x_(j) or frame 156.

The center frame (e.g., frame 156) is determined by hashing the videosignal (or portions thereof). Thus, the locations of the centers of eachregion are determined by the inherent characteristics of the signalitself.

Alternatively, when embedding the watermark, the center frame 156 mightnot be the exact frame determined by the hashing. Thus, the region maybe offset from the frame determined by the hashing. This may be called“offset centering” of the region. The degree of the offset may bedetermined pseudorandomly, mathematically, via hashing, and/or it may befixed. Assuming that the detection system is capably of reliablydetecting the watermarks, this offset centering is acceptable and withinthe scope of the exemplary video watermaker described herein. It isparticularly acceptable because it adds the robustness of the exemplaryvideo watermaker.

Full Mark Zone (FMZ) of the Plateau. The watermark is fully encoded inthe frames surrounding the center frame 156. Theses surrounding framesform a zone called a full mark zone (FMZ) 152. The FMZ 152 ranges fromframe x_(j−z) to x_(j+z) in FIG. 1B, where 2z+1 is the length of thezone. In other words, the full mark zone (such as FMZ 152) includes zframes before the center frame 156 and z frames before the center frame.With reference herein to the FMZs and the plateau regions, the terms“fully encoded,” “gradient encoding,” and “partially encoded” refer tothe relative intensity (e.g., relative strength or scale) with which themark is encoded. “Fully encoded” is full relative intensity.

Generally, fully encoding the watermark in the FMZ 152 of the plateauregion 150 increases the robustness of the watermark encoded by theexemplary video watermaker. Primarily, it reduces (or eliminates) theeffect of desychronization on a video signal (such as signal 144).

Edges of the Plateau. The plateau region 150 includes gradient edges 154a and 154 b in FIG. 1B. On the leading side of the FMZ 152 is leadingedge 154 a. On the trailing side of the FMZ 152 is trailing edge 154 b.Within these edges the watermark is partially encoded. More precisely,the watermark is gradiently encoded, which is either increasingly ordecreasingly encoded.

For leading edge 154 a, the relative intensity with which the mark isencoded increases with successive frames. The leading edge 154 a rangesfrom frame x_(j−k) to x_(j−z) in FIG. 1B, where k−z is the length ofthis edge.

Typically, the trailing edge 154 b is the mirror image of leading edge154 a. For trailing edge 154 b, the relative intensity with which themark is encoded decreases with successive frames. The trailing edge 154b ranges from frame x_(j+z) to x_(j+k) in FIG. 1B, where k−z is thelength of this edge.

Alternatively, the values of k and z on either size of the center neednot be equivalent. In particular, the value of k−z may differ for theleading and trailing edges (e.g., edges 154 a and 154 b). In thissituation, the edges are not mirror images of each other. However, thiswould make the “center” of the region be off the true center. Assumingthat the detection system is capably of reliably detecting thewatermarks, this type of approximate centering is acceptable and withinthe scope of the exemplary video watermaker described herein. It isparticularly acceptable because it adds the robustness of the exemplaryvideo watermaker.

As introduced in the Background section, a malicious attacker can easilyfind abrupt changes in the intensity of consecutive video frames. Suchabrupt changes are common in traditional video watermarking. Thesechanges produce a perceptible flicker effect.

The gradient edges (e.g., edges 154 a and 154 b) of the plateau region150 give the region's boundaries a softer transition between markedframes and unmarked frames. The smooth transition of the gradient edgesis significantly less perceptible—statistically and visually—than the“flicker” of traditional video watermarking.

Generally, gradient encoding the watermark in the FMZ 152 of the plateauregion 150 increases the robustness of the watermark encoded by theexemplary video watermaker. Primarily, it reduces (or eliminates) theeffect of flicker on a video signal (such as signal 144).

Shape of Plateau Regions. The shape of a plateau region isrepresentative of the relative intensity of the watermarking encoding ofthat region. The shapes of the plateau regions illustrated herein (inparticular, in FIGS. 1B and 1C) are only examples. The name “plateau” isintended to be a convenient label that evokes an image of the overallgeneral shape of the region. However, the shapes of the plateau regionsare not limited to those illustrated herein or to the shape of a literalplateau.

With respect to the shape of a plateau, it may take nearly any shape.The range of shapes are bounded by characteristics described in thelanguage claimed herein. Examples of such characteristics of a plateauregions generated by the exemplary video watermaker include:

-   -   one or more frames within the region is fully encoded relative        to other frames (in particular, unmarked frames);    -   one or more frames within the region are gradiently encoded        relative to the fully encoded frames and the unmarked frames.

With these characteristics in mind, a plateau region generated by theexemplary video watermaker may be described, for example, with thefollowing shape descriptions: bump, convex, gibbous, bulge, lump, hump,bulbous, mountain-shaped, peak, mound, mesa, hill, knoll, hillock,butte, drumlin, hummock, dune, tussock, molehill, anthill, dome, arch,hemisphere, half-circle, trapezoid, and the like.

Watermark Detection with the Plateau Regions. A watermark detectionsystem of the exemplary video watermaker processes a subject videosignal. Typically, before such detection, it is unknown whether suchsignal includes a watermark. Likewise, it is unknown whether the subjectsignal has been modified from the original signal. Such modification maybe intentional or unintentional.

Using hashing techniques, the watermark detection system approximatelylocates the center of each plateau region. More precisely, it locatesframes that it considers the centers of their regions. These frames arecalled “detection frames” herein. To find the detection frames, thewatermark detection system uses the same technique as the embeddingsystem uses to locate the regions' center frames.

Typically, the subject video signal itself is hashed to find thedetection frames. If the subject signal remains unmodified from theoriginal, then the detection system finds the actual center of eachregion. If the subject signal has been modified, then the “centers”found by the detection systems approximate the centers of the regions.

However, the watermark detection system need not find the actual centerof a region to detect a watermark encoded within the region. Since thewatermark is encoded in each of the frames of the plateau region, thewatermark detection system is likely to find the embedded mark anywherewithin the region. However, if the approximate center determined by thedetection system falls within a full mark zone (FMZ), then it is morelikely to detect the watermark then if the if the approximate centerfalls within a boundary edge.

For example, assume that video signal 144 of FIG. 1B is a subject signaland that it has been de-synchronized (intentionally or unintentionally).The actual center of the signal before such de-synchronization was frame156. However, the approximate center—determined by hashing thede-synchronized subject signal—may be frame 162, frame 164, or frame166. These frames are indicated by dashed-dotted arrows pointing to theencoded signal of the region 150 of FIG. 1B.

Frame 162 is just off-center from the center frame 156. Like the centerframe and every other frame in the FMZ 152, it is fully encoded with thewatermark. Consequently, the detection system is highly likely to detectthe encoded watermark within frame 162.

Frame 164 is more off-center from the center frame 156 than frame 162,but it is still within the FMZ 152. Similarly, the detection system ishighly likely to detect the encoded watermark within frame 164.

Frame 166 is significantly off-center from the center frame 156. So muchso, that it falls outside the FMZ 152. However, it still is within theplateau region 150. Specifically, it is within trailing edge 154 b.Although the watermark is not fully encoded within this edge, it isstill partially encoded in frame 166 of edge 156 b. Consequently, thereis a possibility that the detection system may detect the encodedwatermark within frame 166.

However, since the mark is only partially encoded there is an increasedpossibility of missing the watermark. In this situation, the video hasmost likely been de-synched sufficiently to produce a modified videothat is perceptually different from the original video.

Conventional watermark detection encodes the marks in isolated frames.With FIG. 1B, if the mark was only encoded in frame 156, the de-synchattack would cause the conventional watermark detection to miss frame156 because it would be looking for the mark in frames 162 or 164.

Multiple Regions

FIG. 1C is a diagram illustrating an example video signal 146 havingmultiple watermarked plateau regions 150 a-150 k in accordance with theexemplary video watermaker. The center of the plateau regions 150 a-150k is defined (i.e., located) by a hash value. Multiple hash values aredetermined by hashing the signal itself.

As a consequence of the almost uniform distributed nature of hashvalues, the plateau regions are distributed throughout the signal in analmost uniform and manner that appears random. Moreover, as illustratedin FIG. 1C, the size and shape of the regions may be varied.

Partially and Fully Blind Approaches

The exemplary video watermaker may be implemented via, at least, twoapproaches: “Partially Blind” and “Fully Blind.” Both are broadly shownin FIGS. 3-6.

Partially Blind Approach:

In the partially blind approach, the exemplary video watermaker selectsmultiple frames of the video. These selected frames will be the centerframes of plateau regions. To select these center frames, the exemplaryvideo watermaker may randomly or pseudorandomly select them. The centerframes define the center of the plateau region for embedding marks.

Once the frames are selected, the exemplary video watermakerhashes thoseselected frames and stores their hash values. These will be called the“center frame hash values” herein. These values identify the locationsof center frames for watermark embedding. These hash values may use asecret key; thus, may be pseudorandom also.

Once these hash values are found, the watermarks are inserted in theplateau regions of the selected center frames. The hash values arestored in a data storage. The locations of the center frames are notstored. Rather, the hash values of those center frames are stored.

The hash values are sent as side information to the watermark detectionsystem. For this approach, it is assumed that the watermark detectionsystem has secure access to the stored hash values of the center frames.The hash values may be cryptographically transmitted to the detectionsystem along with or separate from the video signal being examined bythe system. Regardless, this approach assumes that users (and attackers)do not have access to them.

For example, the hash values may be sent to a software video player on acomputer (or to a DVD player) secretly and the user does not know whatthese hash values are. Another example could be cryptographic encryptionof these hash values via a secret key and transmitting them secretly orspreading them around video data in a secret manner such that attackerswould not be able to find them.

At the detector end, the hash value of each frame is found and comparedto the hash values in the stored list of hash values. For the frameswhose hash values substantially match an entry in the list, the detectorlooks for watermark in that frame. Those frames are called the“detection frames.” Alternatively, it looks for the watermarks inframes—whose hash values does the match—surrounding a detection frame.This is done for each detection frame. The detection frames effectively“synch” the watermark detection process.

The term partially blind comes from the fact that additional informationis needed at the detector side, meaning it is not completely blind. Butit is not non-blind as well. A non-blind watermarking scheme requiresthe original unmarked video to be present at the detector end.

With this approach, an attacker may determine the locations of thedetection frames. To find the detection frames, the attacker only needsto determine the hashing function and gain access to the list of hashvalues (for the center frames).

Fully Blind Approach:

The fully blind approach is the same as the partially blind approach,except that there is no list of center frame hash values. Thus, thewatermark detection system has no list of hash values to compare to thehash of the frames of a subject video signal. Instead, the watermarkdetection system calculates the hash values based upon the subject videosignal.

A comparison between the hash values and randomly generated binaryvectors is made in order to decide if the watermark is going to beembedded or not. At the decoder side, the same operation is carried outusing the same secret key.

By using a secret key, the fully blind approach generates a set ofrandom binary sequence α. This might be, for example, 10 random bits.Then the hash value of each frame within the video sequence is computed,call these h_(i), where i indexes frame number.

As an example, assume that these hash values are of length 100. Randomlychosen 10 locations from a length 100 vector are used and bits fromthose locations are collected (for each frame independently) toconstruct the “symbol” of that frame (in other words, the “symbol” ofeach frame is generated from the hash value of that frame by randomprojection, in this particular example the projection ration is10/100=0.1). Call the symbols s_(i), where i indexes the frames. Oncethe symbol of each frame is computed, it is compared with the initiallyrandomly found vector (i.e., one compares s_(i) with a for all i. Oncethere is an exact match, the watermark is embedded around it in aplateau manner.

This approach is random in the following aspects: first, vector a isgenerated randomly; second, each value h_(i) is the hash value of framei. In addition, by the definition of the hash function, they are alsogenerated randomly, then generation of s_(i) from h_(i) are also done ina random manner. Secret key K is used as the seed of the random numbergenerator in all these steps.

At the detector end, it has access to K. It does the same as above tofind the frame locations. It generates a; it generates hash values; andapplies random projection. It finds the detection frames, which may ormay not be identical to the center frames.

The same secret key and hash function is used for hashing the centerframe for embedding as is used for hashing the detection frames fordetecting. Therefore, if the original unmarked signal and the subjectsignal are perceptually identical, then the hash values of the frameswill be identical or nearly so.

The unmarked signal and subject signal will be perceptually identical isthe subject signal if an unmodified version of the marked signal.Furthermore, the unmarked signal and subject signal will be perceptuallyidentical if the subject signal is a modified version (e.g., an attackedversion) of the marked signal, but such modification did notperceptually modify the signal.

Thus, to determine where the detection frames are located, an attackermust know the secret key and the hash function. This approach isparticularly appropriate when there is great concern about maliciousattacks.

In both approaches, the centers of regions around which watermark isgoing to be embedded are determined. This task may be carried out viausing robust image hash functions. In partially blind approach, the hashvalues are used to determine the locations of the watermarks at thedecoder; thus, partially blind approach is termed to be “partiallyblind”.

In fully blind approach, a comparison between the hash values andrandomly generated binary vectors is made in order to decide if thewatermark is going to be embedded or not. At the decoder side, the sameoperation is carried out using the same secret key; therefore, the fullyblind approach is a completely blind approach.

Exemplary Video Watermarking Architecture

FIG. 2 shows a digital goods production and distribution architecture200 (e.g., video watermarking architecture 200) having a content (e.g.,video) producer/provider 222 that produces original content anddistributes the content over a network 224 to a client 226. The contentproducer/provider 222 has a content storage 230 to store digital goodscontaining original content. The content producer 222 has a watermarkembedding system 232 to sign the digital signals (e.g., video signals)with a watermark that uniquely identifies the content as original. Thewatermark embedding system 232 may be implemented as a standaloneprocess or incorporated into other applications or an operating system.

The watermark embedding system 232 applies the watermark to a digitalsignal (e.g., video signal) from the content storage 230. Typically, thewatermark identifies the content producer 222, providing a signaturethat is embedded in the signal and cannot be cleanly removed.

The content producer/provider 222 has a distribution server 234 thatdistributes the watermarked content over the network 224 (e.g., theInternet). A signal with a watermark embedded therein represents to arecipient that the signal is being distributed in accordance with thecopyright authority of the content producer/provider 222. The server 234may further compress and/or encrypt the content conventional compressionand encryption techniques prior to distributing the content over thenetwork 224.

Typically, the client 226 is equipped with a processor 240, a memory242, and one or more content output devices 244 (e.g., television,display, sound card, speakers, etc.). The processor 240 runs varioustools to process the marked signal, such as tools to decompress thesignal, decrypt the date, filter the content, and/or apply signalcontrols (tone, volume, etc.). The memory 242 stores an operating system250 (such as a Microsoft® Windows 2000® operating system), whichexecutes on the processor. The client 226 may be embodied in a manydifferent ways, including a computer, a handheld entertainment device, aset-top box, a television, an appliance, and so forth.

The operating system 250 implements a client-side watermark detectingsystem 252 to detect watermarks in the digital signal and a contentloader 254 (e.g., multimedia player, audio player) to facilitate the useof content through the content output device(s) 244. If the watermark ispresent, the client can identify its copyright and other associatedinformation.

The operating system 250 and/or processor 240 may be configured toenforce certain rules imposed by the content producer/provider (orcopyright owner). For instance, the operating system and/or processormay be configured to reject fake or copied content that does not possessa valid watermark. In another example, the system could load unverifiedcontent with a reduced level of fidelity.

Exemplary Video Watermarking Embedding System

FIG. 3 shows an exemplary video watermark embedding system 300, which isan example of an embodiment of a portion of the video watermarkingarchitecture 200. The watermark embedding system 232 is an example ofthe exemplary video watermark embedding system 300.

The watermark embedding system 300 includes a signal obtainer 310, aregion locator 320, a region definer 330, and a region marker 340.

The signal obtainer 310 obtains an unmarked video signal 305. It mayobtain the signal from nearly any source, such as a storage device orover a network communications link.

The region locator 320 determines the locations of the regions withinthe signal 305. To determine the region locations, the system may firstdetermine the center frame (such as frame 156 of FIG. 1B) of eachregion. It may determine the center frames pseudorandomly. After that,it finds the hash value of the center frames. These hash valueseffectively identify the locations of the center frames. Thus, thelocations of the centers of each region are determined (via the theirhash values) by the inherent characteristics of the signal itself.Alternatively, the center frame of a region may be offset from theinitial frame determined by the region locator 320.

The region definer 330 defines the parameters of the regions. Suchparameters include region length (2k+1) and FMZ length (2z+1). Theseparameters may fixed for all regions of all signals. They may be fixedfor all regions, but varied per signals. They may be varied for allregions of all signals. Generally speaking, the parameters may bemanipulated to adjust several factors including watermark detectionprobability and robustness.

The region marker 340 watermarks the regions of the signal in a plateaumanner. The frames of the FMZ are fully encoded while the frames of theboundary edges are gradiently encoded. The region marker may employtraditional or new watermarking techniques particularly those designedfor image watermarking. It may employ spread-spectrum, QIM, or otherwatermarking techniques. This marked video may be publicly distributedto consumers and clients.

The functions of aforementioned components of the exemplary videowatermark embedding system 300 of FIG. 3 are explained in more detailabove and below.

Methodological Implementation of the Exemplary Video Watermark Embedder

FIG. 4 shows the methodological implementation of the exemplary videowatermark embedding system 300 (or some portion thereof). Morespecifically, this figure shows the methodological implementation ofwatermark embedding of the exemplary video watermaker. Thismethodological implementation may be performed in software, hardware, ora combination thereof.

At 410 of FIG. 4, the exemplary video watermaker obtains a video signal.Specifically, this signal is unmarked. At 412, it determines thelocations of the plateau regions within the signal. The region locationsare determined by pseudorandomly selecting them (using a secret key).The hash values of selected frame is taken and stored. These values arestored. For the partially blind approach, these values are available tothe decoding system. For example, they may accompany the signal in acryptographic envelope or they come via a separate and secure mechanism.

At 414, the exemplary video watermaker defines the parameters of theregions. At 416, it watermarks the regions of the signal in a plateaumanner. At 418, the process ends.

Exemplary Video Watermarking Detecting System

FIG. 5 shows an exemplary video watermark detecting system 500, which isan example of an embodiment of a portion of the video watermarkingarchitecture 200. The watermark detecting system 252 is an example ofthe exemplary video watermark embedding system 500.

The watermark detecting system 500 includes a signal obtainer 510, aframe locator 520, and a watermark detector 530.

The signal obtainer 510 obtains a subject video signal 505. It mayobtain the signal from nearly any source, such as a storage device orover a network communications link. Typically, it is unknown whether thesubject video signal 505 is marked and whether it has been modified.

The frame locator 520 determines the locations of the “detection frames”of the subject signal 505. The detection frames are the selected framesof the signal that the system will attempt to detect the watermark. Thesystem may also attempt to locate the watermark in frames surroundingthe selected detection frames.

For the both the partially and fully blind approaches, the frame locator520 uses the same hashing technique (including the same secret key) tofind the hash values of frames of the subject signal as the watermarkembedding system 300 (specifically, the region locator 320) used for thecenter frame of a region.

For the partially blind approach, the frame locator 520 compares thejust calculated hash values of each frame of the subject video to thestored list of hash values (which was originally generated by thewatermark embedding system 300). Each exact or substantial match isdesignated a “detection frame.”

For the fully blind approach, the frame locator 520 selects the detectsframes by doing a comparison between the hash values and randomlygenerated binary vectors is made in order to decide if the watermark isgoing to be embedded or not.

The watermark detector 530 determines whether watermarks exist in thedetection frames. The watermark detector employs the detection techniquethat corresponds the watermarking technique employed by the watermarkembedding system 300. The watermark detector 530 indicates whether thewatermark is present.

The functions of aforementioned components of the exemplary videowatermark detecting system 500 of FIG. 5 are explained in more detailabove and below.

Methodological Implementation of the Exemplary Video Watermark Detection

FIG. 6 show methodological implementation of the exemplary videowatermark detecting system 500 (or some portion thereof). Morespecifically, this figure shows the methodological implementation ofwatermark detecting of the exemplary video watermaker. Thismethodological implementation may be performed in software, hardware, ora combination thereof.

At 610 of FIG. 6, the exemplary video watermaker obtains a subject videosignal. Typically, it is unknown whether the subject video signal 505 ismarked and whether it has been modified. At 612, it determines thelocations of the “detection frames” of the subject signal. At 614, theexemplary video watermaker determines whether watermarks exist in thedetection frames. The watermark detector employs the detection techniquethat corresponds the watermarking technique employed by the watermarkembedding methodological implementation of FIG. 4. Typically, it alsoindicates whether the watermark is present. Such indication may bestored, displayed, printed, etc. The process ends at 616.

Other Implementation Details

The “partially blind” approach is partially blind in the sense that itrequires the hash values to be known at the decoder side. The “fullyblind” approach is completely blind and relies on the fact that thecollision probability of the hash values should be low and approximateuniform distribution should be achieved.

Given the input video {X_(i)} of length N, the exemplary videowatermaker embeds watermarks {W_(j)}, 1≦j≦M, each of length L_(w), at Mdifferent places. The secret key K is the seed of the pseudorandomnumber generator in all the randomized tasks in both approaches. LetH(·) denote the hash function to be used that produces a hash of lengthL. Let a(k) denote the k-th element of an input vector a. Let ƒ be acontinuous monotonic even function defined on real numbers such thatƒ(0)=1, ƒ(x)=0 for |x|≧1 and ƒ(|x|)≧ƒ(|x|+|ε|), x, εεR. Let {Y_(i)} bean input to the watermark decoder.

Partially Blind Approach

Watermark Encoding: Pseudorandomly pick the frames X_(i) ₁ , X_(i) ₂ , .. . X_(i) _(M) to be the center of the watermark regions, where i_(j)ε{1,2, . . . , N}, 1≦j≦M and |i_(j)−i_(k)|≧K for all j≠k, 1≧j,k≧M. HereKεZ⁺ may be a user-selected parameter. Assume w.l.o.g. that i₁<i₂< . . .<i_(M).

Find H(X_(i) _(j) ), 1≦j≦M.

Find the semi-global statistics {μ_(i) _(j) } and their quantizedvalues, {{circumflex over (μ)}_(i) _(j) } after watermark embedding for1≦j≦M.

Compute sign vectors s_(i) _(j) for 1≦j≦M such that s_(i) _(j) (k)=1 if{circumflex over (μ)}_(i) _(j) (k)≧μ_(i) _(j) (k) and 0 otherwise.

Pseudorandomly find the watermarking neighborhood widths {W_(i) _(j) }such that W_(i) _(j) εZ⁺, 1≦j≦M and furthermore W_(i) _(j) +W_(i) _(j+1)≦K for 1≦j≦M−1 is satisfied.

For frame X_(i) _(j) _(+k) _(j) , embed the watermark w_(j) usingquantization index modulation (QIM) watermarking and employingΔ_(J)ƒ(k_(J)/W_(i) _(j) ) as the quantizer step size and s_(i) _(j) asthe sign vector where 1≦j≦M, −W_(i) _(j) ≦k_(j)≦W_(i) _(j) and {Δ_(j)}are user entered positive parameters.

Watermark Decoding: Given the input video {Y_(i)} of length N_(Y),compute hash values H(Y_(i)) for all i.

Find frames {Y_(ĩ) _(j) } for all 1≦j≦M such that d(H(Y_(ĩ) _(j) ),H(X_(i) _(j) ))≦d(H(Y_(k)), H(X_(i) _(j) )) for all 1≦k≦N_(Y).

For each frame Y_(ĩ) _(j) _(+k) _(j) , −α_(j)W_(i) _(j)≦k_(j)≦α_(j)W_(i) _(j) , 1≦j≦M, carry out watermark decoding usingquantization index modulation (QIM) watermarking and employingΔ_(j)ƒ(k_(j)/W_(i) _(j) ) as the quantizer step size. Here 0<α_(j)≦1 areuser determined parameters. Let the decoded vectors be {w_(D,k) _(j) }.

Given {w_(D,k) _(j) }, find {w_(D,j)} using the majority rule:w_(D,j)(l)=1 if${\sum\limits_{k_{j} = {- {{round}{({\alpha_{j}W_{i_{j}}})}}}}^{{round}{({\alpha_{j}W_{i_{j}}})}}{w_{D,k_{j}}(l)}} > {{round}\left( {\alpha_{j}W_{i_{j}}} \right)}$and 0 otherwise. Here round(.) denotes rounding to integer operation andw_(D,) _(J) is the resulting decoded vector that is derived from theneighborhood of Y_(ĩ) _(j) .

Declare that the watermark is present if max_(j) d(w_(D,j), w_(j))<T;not present otherwise.

Fully Blind Approach

Watermark Encoding: Generate M random binary vectors h_(j), 1≦j≦M oflength {tilde over (L)} where {tilde over (L)}<<L.

Find H(X_(i)), 1≦i≦N.

Find h_(i) ^(X), 1≦i≦N of length {tilde over (L)}such that h_(i) ^(X)(k)is the l_(k)-th element of H(X_(i)) for 1≦k≦{tilde over (L)} and {l₁,l₂, . . . , l_({tilde over (L)})} is a random subset of {1,2, . . . , L}.

For each j, 1≦j≦M, find frames X_(j) _(k) such that h_(j) _(k)^(X)=h_(j), 1≦k≦N_(j) for each j, 1≦j≦M. Here N_(j) is the number of theplaces where the random binary vector h_(j) matches the hash values ofthe input video.

For each X_(j) _(k) , compute the semi-global statistics, theirquantized values and the corresponding sign vectors; randomly find thewatermarking neighborhood widths such that there is no overlap betweendifferent neighborhoods (similar to portions of the encoding part ofpartially blind approach). Let s_(j) _(k) be the sign vector for X_(j)_(k) and W_(j) be the watermarking neighborhood width for X_(j) _(k)(same for all k for a particular j).

For frame X_(j) _(k) _(+l) _(j) , embed the watermark w_(j) by using thecorresponding sign vector s_(j) _(k) and the quantization step sizeΔ_(j)ƒ(l_(j)/W_(j)), −W_(j)≦l_(j)≦W_(j), 1≦k≦N_(j) for each j, 1≦j≦M.Use QIM watermarking for watermark embedding. Here quantizer step sizes{Δ_(j)} are user entered positive parameters.

Watermark Decoding: Given the input video {Y_(i)} of length N_(Y),compute hash values H(Y_(i)) for all i.

Find h_(i) ^(Y), 1≦i≦N_(Y) of length {tilde over (L)} such that h_(i)^(Y)(k) is the l_(k)-th element of H(Y_(i)) for 1≦k≦{tilde over (L)} and{l₁,l₂, . . . , l_({tilde over (L)})} is the same subset of {1,2, . . ., L} found in above in encoding.

For each j, 1≦j≦M, find frames Y_(j) _(k) such that h_(j) _(k)^(Y)=h_(j), 1≦{tilde over (k)}≦Ñ_(j), 1≦j≦M. Here Ñ_(j) is the number ofthe places where the random binary vector h_(j) matches the hash valuesof the input video.

For each frame Y_(j) _({tilde over (k)}) _(+l) _(j) ,−α_(j)W_(j)≦l_(j)≦α_(j)W_(j), 1≦{tilde over (k)}≦Ñ_(j), 1≦j≦M, carry outwatermark decoding using QIM watermarking and employingΔ_(j)ƒ(l_(j)/W_(j)) as the quantizer step size. Here 0<α_(j)≦1 are userdetermined parameters. Let the decoded vectors be {W_(D,j) _(k) ,l_(j)}.

Given {W_(D,j) _(k) , l_(j)}, find {W_(D,j)} using the majority rule:W_(D,j)(m)=1 if${\sum\limits_{\overset{\sim}{k} = 1}^{{\overset{\sim}{N}}_{j}}{\sum\limits_{l_{j} = {- {{round}{({\alpha_{j}W_{j}})}}}}^{{round}{({\alpha_{j}W_{j}})}}{W_{D,j_{\overset{\sim}{k}},l_{j}}(m)}}} > {{\overset{\sim}{N}}_{j}{{round}\left( {\alpha_{j}W_{j}} \right)}}$and 0 otherwise. Here round(.) denotes rounding to integer operation andW_(D,) _(J) is the resulting decoded vector that is derived from theneighborhood of Y_(j) _(k) over all possible {tilde over (k)}ε{1, 2, . .. , Ñ_(j)}.

Declare that the watermark is present if max_(j) d(w_(D,j), w_(j))<T;not present otherwise.

Decreasing Visual Artifacts

In both approaches, for a given center frame, the watermark are embeddedin a neighborhood around it. This may be done using QIM watermarking.During this process, the sign vectors, that are derived from the centerframe, may play a role in terms of decreasing visual artifacts. For aparticular statistic, if one quantizes to a higher value for a givenframe and if one quantizes to a lower value for a neighboring frame,there will be a slight “flickering effect.” In order to minimize thiseffect, a fixed sign vector may be employed. That vector is used for thewhole range of neighborhood.

Exemplary Computing System and Environment

FIG. 7 illustrates an example of a suitable computing environment 900within which an exemplary video watermaker, as described herein, may beimplemented (either fully or partially). The computing environment 900may be utilized in the computer and network architectures describedherein.

The exemplary computing environment 900 is only one example of acomputing environment and is not intended to suggest any limitation asto the scope of use or functionality of the computer and networkarchitectures. Neither should the computing environment 900 beinterpreted as having any dependency or requirement relating to any oneor combination of components illustrated in the exemplary computingenvironment 900.

The exemplary video watermaker may be implemented with numerous othergeneral purpose or special purpose computing system environments orconfigurations. Examples of well known computing systems, environments,and/or configurations that may be suitable for use include, but are notlimited to, personal computers, server computers, thin clients, thickclients, hand-held or laptop devices, multiprocessor systems,microprocessor-based systems, set top boxes, programmable consumerelectronics, network PCs, minicomputers, mainframe computers,distributed computing environments that include any of the above systemsor devices, and the like.

The exemplary video watermaker may be described in the general contextof computer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types. Theexemplary video watermaker may also be practiced in distributedcomputing environments where tasks are performed by remote processingdevices that are linked through a communications network. In adistributed computing environment, program modules may be located inboth local and remote computer storage media including memory storagedevices.

The computing environment 900 includes a general-purpose computingdevice in the form of a computer 902. The components of computer 902 caninclude, by are not limited to, one or more processors or processingunits 904, a system memory 906, and a system bus 908 that couplesvarious system components including the processor 904 to the systemmemory 906.

The system bus 908 represents one or more of any of several types of busstructures, including a memory bus or memory controller, a peripheralbus, an accelerated graphics port, and a processor or local bus usingany of a variety of bus architectures. By way of example, sucharchitectures can include an Industry Standard Architecture (ISA) bus, aMicro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, aVideo Electronics Standards Association (VESA) local bus, and aPeripheral Component Interconnects (PCI) bus also known as a Mezzaninebus.

Computer 902 typically includes a variety of computer readable media.Such media can be any available media that is accessible by computer 902and includes both volatile and non-volatile media, removable andnon-removable media.

The system memory 906 includes computer readable media in the form ofvolatile memory, such as random access memory (RAM) 910, and/ornon-volatile memory, such as read only memory (ROM) 912. A basicinput/output system (BIOS) 914, containing the basic routines that helpto transfer information between elements within computer 902, such asduring start-up, is stored in ROM 912. RAM 910 typically contains dataand/or program modules that are immediately accessible to and/orpresently operated on by the processing unit 904.

Computer 902 may also include other removable/non-removable,volatile/non-volatile computer storage media. By way of example, FIG. 7illustrates a hard disk drive 916 for reading from and writing to anon-removable, non-volatile magnetic media (not shown), a magnetic diskdrive 918 for reading from and writing to a removable, non-volatilemagnetic disk 920 (e.g., a “floppy disk”), and an optical disk drive 922for reading from and/or writing to a removable, non-volatile opticaldisk 924 such as a CD-ROM, DVD-ROM, or other optical media. The harddisk drive 916, magnetic disk drive 918, and optical disk drive 922 areeach connected to the system bus 908 by one or more data mediainterfaces 926. Alternatively, the hard disk drive 916, magnetic diskdrive 918, and optical disk drive 922 can be connected to the system bus908 by one or more interfaces (not shown).

The disk drives and their associated computer-readable media providenon-volatile storage of computer readable instructions, data structures,program modules, and other data for computer 902. Although the exampleillustrates a hard disk 916, a removable magnetic disk 920, and aremovable optical disk 924, it is to be appreciated that other types ofcomputer readable media which can store data that is accessible by acomputer, such as magnetic cassettes or other magnetic storage devices,flash memory cards, CD-ROM, digital versatile disks (DVD) or otheroptical storage, random access memories (RAM), read only memories (ROM),electrically erasable programmable read-only memory (EEPROM), and thelike, can also be utilized to implement the exemplary computing systemand environment.

Any number of program modules can be stored on the hard disk 916,magnetic disk 920, optical disk 924, ROM 912, and/or RAM 910, includingby way of example, an operating system 926, one or more applicationprograms 928, other program modules 930, and program data 932. Each ofsuch operating system 926, one or more application programs 928, otherprogram modules 930, and program data 932 (or some combination thereof)may include an embodiment of a signal obtainer, a region locator, aregion definer, a region marker, a signal marker, a frame locator, asynchronizer, and a watermark detector.

A user can enter commands and information into computer 902 via inputdevices such as a keyboard 934 and a pointing device 936 (e.g., a“mouse”). Other input devices 938 (not shown specifically) may include amicrophone, joystick, game pad, satellite dish, serial port, scanner,and/or the like. These and other input devices are connected to theprocessing unit 904 via input/output interfaces 940 that are coupled tothe system bus 908, but may be connected by other interface and busstructures, such as a parallel port, game port, or a universal serialbus (USB).

A monitor 942 or other type of display device can also be connected tothe system bus 908 via an interface, such as a video adapter 944. Inaddition to the monitor 942, other output peripheral devices can includecomponents such as speakers (not shown) and a printer 946 which can beconnected to computer 902 via the input/output interfaces 940.

Computer 902 can operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computingdevice 948. By way of example, the remote computing device 948 can be apersonal computer, portable computer, a server, a router, a networkcomputer, a peer device or other common network node, and the like. Theremote computing device 948 is illustrated as a portable computer thatcan include many or all of the elements and features described hereinrelative to computer 902.

Logical connections between computer 902 and the remote computer 948 aredepicted as a local area network (LAN) 950 and a general wide areanetwork (WAN) 952. Such networking environments are commonplace inoffices, enterprise-wide computer networks, intranets, and the Internet.

When implemented in a LAN networking environment, the computer 902 isconnected to a local network 950 via a network interface or adapter 954.When implemented in a WAN networking environment, the computer 902typically includes a modem 956 or other means for establishingcommunications over the wide network 952. The modem 956, which can beinternal or external to computer 902, can be connected to the system bus908 via the input/output interfaces 940 or other appropriate mechanisms.It is to be appreciated that the illustrated network connections areexemplary and that other means of establishing communication link(s)between the computers 902 and 948 can be employed.

In a networked environment, such as that illustrated with computingenvironment 900, program modules depicted relative to the computer 902,or portions thereof, may be stored in a remote memory storage device. Byway of example, remote application programs 958 reside on a memorydevice of remote computer 948. For purposes of illustration, applicationprograms and other executable program components such as the operatingsystem are illustrated herein as discrete blocks, although it isrecognized that such programs and components reside at various times indifferent storage components of the computing device 902, and areexecuted by the data processor(s) of the computer.

Computer-Executable Instructions

An implementation of an exemplary video watermaker may be described inthe general context of computer-executable instructions, such as programmodules, executed by one or more computers or other devices. Generally,program modules include routines, programs, objects, components, datastructures, etc. that perform particular tasks or implement particularabstract data types. Typically, the functionality of the program modulesmay be combined or distributed as desired in various embodiments.

Exemplary Operating Environment

FIG. 7 illustrates an example of a suitable operating environment 900 inwhich an exemplary video watermaker may be implemented. Specifically,the exemplary video watermaker(s) described herein may be implemented(wholly or in part) by any program modules 928-930 and/or operatingsystem 926 in FIG. 7 or a portion thereof.

The operating environment is only an example of a suitable operatingenvironment and is not intended to suggest any limitation as to thescope or use of functionality of the exemplary video watermaker(s)described herein. Other well known computing systems, environments,and/or configurations that are suitable for use include, but are notlimited to, personal computers (PCs), server computers, hand-held orlaptop devices, multiprocessor systems, microprocessor-based systems,programmable consumer electronics, wireless phones and equipments,general- and special-purpose appliances, application-specific integratedcircuits (ASICs), network PCs, minicomputers, mainframe computers,distributed computing environments that include any of the above systemsor devices, and the like.

Computer Readable Media

An implementation of an exemplary video watermaker may be stored on ortransmitted across some form of computer readable media. Computerreadable media can be any available media that can be accessed by acomputer. By way of example, and not limitation, computer readable mediamay comprise “computer storage media” and “communications media.”

“Computer storage media” include volatile and non-volatile, removableand non-removable media implemented in any method or technology forstorage of information such as computer readable instructions, datastructures, program modules, or other data. Computer storage mediaincludes, but is not limited to, RAM, ROM, EEPROM, flash memory or othermemory technology, CD-ROM, digital versatile disks (DVD) or otheroptical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which canbe used to store the desired information and which can be accessed by acomputer.

“Communication media” typically embodies computer readable instructions,data structures, program modules, or other data in a modulated datasignal, such as carrier wave or other transport mechanism. Communicationmedia also includes any information delivery media.

The term “modulated data signal” means a signal that has one or more ofits characteristics set or changed in such a manner as to encodeinformation in the signal. By way of example, and not limitation,communication media includes wired media such as a wired network ordirect-wired connection, and wireless media such as acoustic, RF,infrared, and other wireless media. Combinations of any of the above arealso included within the scope of computer readable media.

Conclusion

Although the invention has been described in language specific tostructural features and/or methodological steps, it is to be understoodthat the invention defined in the appended claims is not necessarilylimited to the specific features or steps described. Rather, thespecific features and steps are disclosed as preferred forms ofimplementing the invention.

1. A method for facilitating the insertion of information into a videosignal, the method comprising: embedding an information pattern in oneor more regions of the video signal, a region comprising a pluralitysuccessive frames; locating the one or more regions of the video signalby randomly selecting frames of the signal, where each of the regionsincludes at least one of the selected frames; wherein, for one or moreregions, the information pattern is embedded into a frame of a regionwith a relative degree of intensity and one or more frames of thatregion have the patterns embedded therein which have a relativeintensity that differs from the relative intensity of the patternembedded in other frames of the region.
 2. A method as recited in claim1 for facilitating the protection of a video signal, wherein theinformation pattern is a watermark.
 3. A method as recited in claim 1further comprising determining hash values of one or more frames of oneor more of the regions of the video signal.
 4. A method as recited inclaim 1 further comprising: determining hash values of one or moreframes of one or more of the regions of the video signal; persisting thehash values with an association with the video signal.
 5. A method asrecited in claim 1, wherein, for one or more regions, the informationpattern is embedded into a frame of a region in a plateau-shaped manner.6. A method as recited in claim 1, wherein, for one or more regions, therelative intensity of information pattern embedded into a group ofsuccessive frames of the plurality of a region is substantiallyidentical.
 7. A method as recited in claim 1, wherein, for one or moreregions, the relative intensity of information pattern embedded into agroup of successive frames of the plurality of a region is approximatelysimilar.
 8. A method as recited in claim 1, wherein, for one or moreregions, the relative intensity of information pattern embedded into agroup of successive frames of the plurality of a region is gradient. 9.A computer-readable medium having computer-executable instructions that,when executed by a computer, performs a method as recited in claim 1.10. A computer comprising one or more computer-readable media havingcomputer-executable instructions that, when executed by the computer,perform a method as recited in claim
 1. 11. A method for facilitatingthe insertion of information into a video signal, the method comprising:embedding an information pattern in one or more regions of the videosignal, a region comprising a plurality successive frames; specifyingthe dimensions of the one or more regions of the video signal; wherein,for one or more regions, the information pattern is embedded into aframe of a region with a relative degree of intensity and one or moreframes of that region have the patterns embedded therein which have arelative intensity that differs from the relative intensity of thepattern embedded in other frames of the region.
 12. A method forfacilitating the protection of a video signal, the method comprising:locating one or more watermark regions of a video signal for embedding awatermark therein, a region comprising a plurality successive frames;for one or more watermark regions, embedding a watermark in a watermarkregion in an approximate plateau-shaped manner; wherein the locatingcomprises randomly locating the one or more regions by randomlyselecting frames of the signal, where each of the regions includes atleast one of the selected frames.
 13. A method as recited in claim 12,wherein the watermark is embedded into a frame of the region with arelative degree of intensity and one or more frames of that region havea watermark embedded therein which has a relative intensity that differsfrom the relative intensity of the watermark embedded in other frames ofthat region.
 14. A method as recited in claim 12, wherein the locatingcomprises randomly locating the one or more regions.
 15. A method asrecited in claim 12 further comprising determining hash values of one ormore frames of one or more of the regions of the video signal.
 16. Amethod as recited in claim 12 further comprising: determining hashvalues of one or more frames of one or more of the regions of the videosignal; persisting the hash values with an association with the videosignal.
 17. A method as recited in claim 12, wherein, for one or moreregions, the watermark is embedded into a frame of a region in aplateau-shaped manner.
 18. A method as recited in claim 12, wherein, forone or more regions, the relative intensity of the watermark embeddedinto a group of successive frames of the plurality of a region issubstantially identical.
 19. A method as recited in claim 12, wherein,for one or more regions, the relative intensity of the watermarkembedded into a group of successive frames of the plurality of a regionof a region is approximately similar.
 20. A method as recited in claim12, wherein, for one or more regions, the relative intensity of thewatermark embedded into a group of successive frames of the plurality ofa region is gradient.
 21. A computer-readable medium havingcomputer-executable instructions that, when executed by a computer,performs a method as recited in claim
 12. 22. A computer comprising oneor more computer-readable media having computer-executable instructionsthat, when executed by the computer, perform a method as recited inclaim
 12. 23. A method for facilitating the protection of a videosignal, the method comprising: locating one or more watermark regions ofa video signal for embedding a watermark therein, a region comprising aplurality successive frames; for one or more watermark regions,embedding a watermark in a watermark region in an approximateplateau-shaped manner; specifying the dimensions of the one or moreregions of the video signal.
 24. A method for facilitating theprotection of a video signal, the method comprising: fully encoding oneor more frames of a region of a video signal with a watermark, whereinsuch frames are fully encoded with the watermark relative to unmarkedframes; partially encoding one or more frames of the region with thewatermark, wherein such frames are partially encoded with the watermarkrelative to the frames of the fully encoding.
 25. A computer-readablemedium having computer-executable instructions that, when executed by acomputer, performs a method as recited in claim
 24. 26. A computercomprising one or more computer-readable media havingcomputer-executable instructions that, when executed by the computer,perform a method as recited in claim
 24. 27. A method for facilitatingthe protection of a video signal, the method comprising: fully encodingone or more frames of a region of a video signal with a watermark,wherein the frames are fully encoded with the watermark relative tounmarked frames; gradiently encoding at least one plurality ofsuccessive frames of the region with the watermark, wherein theplurality is gradiently encoded with the watermark relative to theunmarked frames and the fully encoded frames of the fully encoding. 28.A computer-readable medium having computer-executable instructions that,when executed by a computer, performs a method as recited in claim 27.29. A computer comprising one or more computer-readable media havingcomputer-executable instructions that, when executed by the computer,perform a method as recited in claim 27.